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ABSTRACT 

This "Research Roundup" focuses on how schools can benefit 
when educational leaders use data to challenge their intuition and 
assumptions in their decision-making. In "Building an Automated Student 
Record System: A Step-by-Step Guide for Local and State Education Agencies," 
Barbara S. Clements provides administrators with a quick but comprehensive 
introduction to the subject, outlining 12 steps, from setting goals to 
reporting the data. "Data-Driven Decision-Making," by the Education 
Commission of -the States, is an issue brief aimed at helping educators come 
to grips with the challenges of the new ESEA accountability requirements. 
"Using Data to Improve Schools, " by the American Association of School 
Administrators, outlines a process whereby good information is used well, 
leading to smarter decisions, defusing the emotion in controversial issues, 
and providing fuel for meaningful dialogue within the educational community. 
Thomas P. Jandris offers principals a primer on making assessment a part of 
their school's culture in "Data-Based Decision-Making: Essentials for 
Principals." Edie L. Holcomb argues, in "Getting Excited About Data: How to 
Combine People, Passion, and Proof," that although data analysis inevitably 
revolves around numbers, which many people regard as cold, abstract, or even 
dehumanizing, human hopes, fears, and passions are close beneath the surface. 
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^ — I j — >| ie public school system 

has always been somewhat 
ambivalent about the role 
of statistical data in edu- 
cation. On the one hand, schools 
generate an abundance of infor- 
mation. Principals, wearily con- 
fronting a steady flow of forms 
they are required to fill out, know 
the list all too well: lunch counts, 
attendance records, test scores, 
discipline referrals, and dozens 
more. 

On the other hand, the system 
has never been quite certain what 
to do with all this information. 
Traditionally, school leaders have 
satisfied their data obligations by 
filing the required numbers in the 
appropriate bureaucratic pigeon- 
holes and moving on to the next 
task. Seldom have they been 
expected to actually use the data 
in making decisions that could 
benefit student achievement. 

Increasingly, however, state- 
mandated accountability systems 
and the recently reauthorized 
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ESEA have forced schools to 
consider ways that data can be 
used for guiding school improve- 
ment. Moreover, the new legisla- 
tion requires schools to 
disaggregate scores by ethnicity, 
disability, gender, socioeconomic 
level, migrant status, and English 
proficiency. Clearly, schools will 
need the capacity to efficiently 
extract and report a wide range of 
information and use k wisely. 

Finding the right approach may 
not be easy. Told to be “data- 
driven,” some school leaders may 
launch a single-minded push to 
raise test scores by any means at 
hand. Others may stray into a 
public relations approach that 
emphasizes spin control over sub- 
stantive improvement. 

The five documents reviewed in 
this Research Roundup find that 
schools benefit when leaders use 
the data to challenge their intu- 
itions and assumptions. Defining 
“data-driven” simply as “the con- 
sistent use of objective informa- 
tion to enhance human 
judgment,” the authors promote 
the use of data not just to satisfy 
legal requirements but also to 
focus action to improve student 
learning. 



IBISST COPY AVAILABLE 




Barbara Clements piovidts 
guidance on establishing auto- 
mated data systems. 

The Education Commission of 
the States examines the data 
practices of exemplary districts. 

The American Association of 
School Administrators explains 
how data can be used to make 
better decisions. 

Thomas Jandris princi- 
pals a primer on making assess- 
ment a part of the school’s 
culture. 

Edie Holcomb explores the 
human side of data use. 
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1398, Jessup, MD 20794-1398. 

Free publication. 877-433-7827. 
Available online at www.nces.ed. 
gov/pubs2000/building. 



With the new ESEA require- 
ments, keeping track of the data 




may turn out to be as big a chal- 
lenge as gathering it. Not surpris- 
ingly, many schools have turned to 
computerized storage-and-retrieval 
systems. This booklet from the 
National Forum on Education 
Statistics provides administrators 
with a quick but comprehensive 
introduction to the subject, outlin- 
ing 12 steps, from setting goals to 
reporting the data. 

Automated systems carry several 
advantages, including cost savings, 
accuracy, flexibility, quicker 
retrieval, and easier reporting. 
However, computerization also 
heightens the potential for data 
proliferation. Therefore, before 
schools make the commitment to 
automate, they should determine 
what information is important, 
how it will be used, and who will 
use it. In most cases, schools will 
be looking for information that 
satisfies accountability require- 
ments, provides feedback on the 
overall health of the school, or 
guides instructional improvement. 

A major concern with any record 
system, especially an automated 
system, is security and privacy. As 
many users of e-mail have discov- 
ered, electronic information can 
quickly spread beyond the intended 
users. Because federal privacy regu- 
lations require that schools provide 
access only to those with a recog- 
nized “need to know,'' policies 
should indicate who has access to 
various kinds of records, what 
security procedures will be used to 
prevent inappropriate access, and 
what penalties will be imposed. 

At the same time, schools should 
develop procedures to ensure that 
students and their parents can 
access personal records, as 
required by law. Finally, procedures 
should be established to discard 
outdated information. 

Although this booklet does not 
provide comparisons or other tech- 
nical guidance on specific software 
or hardware systems or vendors, it 
will help school leaders ask the 
right questions when the time 
comes to make a purchase. 



Education Commission of the 
States. Data-Driven Decision- 
making. Denver: Education 
Commission of the States, 2002. 
Available from: Education 
Commission of the States, 700 
Broadway, Suite 1200, Denver, CO 
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This issue brief from the 
Education Commission of the 
States is one in a series aimed at 
helping educators come to grips 
with the challenges of the new 
ESEA accountability require- 
ments. Based on a study of 
schools using exemplary data 
practices, it provides a brief but 
well-grounded overview. 

First, the paper makes it clear 
that data are not restricted to test 
scores. Schools can also draw on 
demographic statistics, such as 
gender, ethnicity, and number of 
years in the district, as well as 
curriculum content, instructional 
interventions, and “perception” 
data, such as teacher beliefs 
about assessment or community 
attitudes about standards. 
Connecting these different types 
of data permits informed judg- 
ments about which students are 
benefitting from which 
approaches. 

Second, data can be used for a 
wide variety of purposes, includ- 
ing tracking student progress, 
guiding teachers' professional 
development, deciding on 
instructional and curricular inter- 
ventions, creating and assessing 
school improvement plans, and 
allocating district resources. 

Third, data should be used 
throughout the district. At the 
school level, principals can 
make sure data use is embedded 
in everyday practices, provide 
appropriate professional develop- 



ment for staff, and continually 
engage teachers in discussion 
about the data and their 
implications. 

In the exemplary districts dis- 
cussed in this report, student 
populations are characterized by 
low income, high mobility, and 
poor test scores. Their success in 
using data to improve instruction 
demonstrates that schools can 
move from “fixing the blame” to 
“fixing the problem.” 



American Association of School 
Administrators. Using Data to 
Improve Schools. Arlington, Va.: 
American Association of School 
Administrators, 2002. Available 
from: American Association of 
School Administrators, 1801 N. 
Moore St., Arlington, VA 22209. 

Free publication. 703-528-0700; fax 
703-841-1540. Available online at 
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Used correctly, good informa- 
tion leads to smarter decisions, 
defuses the emotion in contro- 
versial issues, and provides fuel 
for meaningful dialogue within 
the educational community. This 
report from AASA outlines a 
process for reaping those 
benefits. 

Success begins with a clear 
sense of purpose. The data 
alone are meaningless unless 
linked to participants’ under- 
standing of what the school is 
trying to accomplish and how it 
plans to accomplish it. For that 
reason, leaders should discour- 
age reflexive or simplistic 
responses to measurements. 

Any data point should gener- 
ate numerous interpretive ques- 
tions before action is taken. 

How does this score compare 
with previous scores? Do differ- 
ent demographic groups score 
differently? Do results reflect 
recent changes in the instruc- 
tional program? Do the results 
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suggest the need for changes? 
The goal is not to make every- 
one a data technician but to cre- 
ate an understanding that 
intelligent use of information 
can move the school closer to 
fulfilling its mission. 

Because public reporting 
requirements ensure that school 
data will attain high visibility, 
the report suggests three strate- 
gies for cultivating community 
involvement and understanding; 

• Engaging the public in dia- 
logue can give schools insights 
into what information is most 
meaningful to the community. 

• Crafting a thoughtful commu- 
nications plan will make sure 
that parents and public get accu- 
rate information in a timely way. 

• Working with the media will 
get the school's message before 
the public. 

The report concludes with a few 
words of caution: Don’t make 
quick decisions without consulting 
all the key players; don’t exalt data 
for its own sake; and prioritize 
the indicators that are most 
important. Most of all, remember 
to continually remind people of 
the purpose behind the data and 
the benefits they will bring. 
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The current interest in assess- 
ment data is motivated by the 
demands of accountability. But 
the real power of data is 
unleashed only when assessment 
is infused throughout a school’s 
culture. This monograph by 
Thomas Jandris, prepared under 
the sponsorship of NAESP and 
the Educational Research Service, 
explores the leadership challenges 
for principals. 

The first step is recognizing the 
need for a comprehensive 
approach. Limiting data use to the 
annual state-wide test will fail to 
harness the full power of assess- 
ment. The report borrows from 
the Japanese kaizen philosophy 
that produces quality through 
continuous incremental improve- 
ments at multiple points along the 
way, rather than a single “inspec- 
tion and correction” at the end of 
the process. 

One of the principal’s key respon- 
sibilities is to build sound assess- 
ment practices into the school’s 
culture. Principals create the right 
conditions when they establish a 
supportive climate, provide time for 
assessment, train teachers to use 
assessment data when planning 
instruction, and state clear expecta- 
tions for performance. 

Assessment scores can have 
multiple meanings, depending on 
who is interpreting the results. To 
maintain coherence, leaders must 
anchor the information to these 



key questions: What do we want 
students to know and be able to 
do? What kind of assessment will 
tell us how students are doing on 
these criteria? How well are they 
performing? What does the 
assessment suggest about ways to 
improve? 

Answers to such questions will 
come only through continuous 
dialogue, debate, and discussion. 
The author recommends using the 
Annenberg Institute’s “Inquiry 
Cycle,” which focuses improve- 
ment efforts into a continuous 
six-step process: 

• Establish the desired outcomes; 

• Define the questions; 

• Collect and organize the data; 

• Make meaning of the data; 

• Take action; and 

• Assess and evaluate actions. 

In addition to providing princi- 
pals with a strong conceptual 
background to guide their use of 
data, the report offers numerous 
concrete examples, a long list of 
resources, and a glossary of 
assessment terms. 
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Web site: www.corwlnpress.com. 



Data analysis inevitably revolves 
around numbers, which many 
people regard as cold, abstract, or 
even dehumanizing. But as Edie 
Holcomb argues in this book, 
human hopes, fears, and passions are 
never far beneath the surface. 

In the end, Holcomb notes, effec- 
tive data use is less a technical problem 
than a cultural problem, and it needs to 
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be approached on that basis. Drawing 
on her experience as staff developer 
and consultant, she provides a road map 
for leaders faced widi the challenge of 
“getting people on board.” 

Teachers have many reasons for 
resisting data analysis. They may lack the 
time or the right kind of training, or 
they may fear that the results will be 
used against them. The key is to help 
teachers connect the data to their pas- 
sion. One way of doing this is through 
the school mission statement. Reminded 
of the mission, teachers can proceed 
more readily to two questions: What 
data do we currendy have that can help 
measure our success? What additional 
data could we use? 



Once teachers begin considering data 
that might help them make decisions, 
they are often overwhelmed by the pos- 
sibilities. Holcomb offers three strategies 
to avoid data paralysis. Her preferred 
approach is constructivist, relying on 
intensive dialogue on questions such as: 
What evidence would demonstrate ful- 
fillment of the mission? Can we find 
more than one indicator for a particular 
goal? What data will verify our assump- 
tions about student learning? What data 
might help resolve smoldering issues in 
the school? 

Holcomb notes that readiness varies, 
but that the important point is to start 
somewhere. Schools not ready to 
make the commitment to a construc- 



tivist approach can focus on compli- 
ance (dealing only with the data 
required by external agencies) or 
even minimalism (using whatever 
data happen to be at hand). 

While Holcomb concedes at the 
start that her book is neither highly 
technical not comprehensive, it pro- 
vides a detailed road map for getting 
teachers involved with, and motivat- 
ed by, data. Readers will find not 
only a coherent conceptual frame- 
work but also useful advice on a vari- 
ety of practical issues, from finding 
time to displaying data in a user- 
friendly way, 

□ 
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